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Abstract Body 

Limit 4 pages single-spaced. 


Background / Context: 

Ratio and proportional relationships are of primary importance during the upper 
elementary and middle school grades (Kilpatrick, Swafford, & Findell, 2001; National Council 
of Teachers of Mathematics, 1989, 2000; National Mathematics Advisory Panel [NMAP], 2008). 
These relationships, along with the interrelated topics of fractions, decimals, and percent provide 
a critical foundation for algebra (NMAP, 2008). Proportionality involves the concept of ratio and 
is central to topics in mathematics such as linear functions, scale drawings, similarity, 
trigonometry, and probability. Proportionality is also at the core of many important concepts in 
the physical and social sciences. As such, the Curriculum and Evaluation Standards for School 
Mathematics (NCTM, 1989) considered proportionality to be “of such great importance that it 
merits whatever time and effort must be expended to assure its careful development” (p. 82). In 
the Common Core State Standards (CCSS; National Governors Association Center for Best 
Practices & Council of Chief State School Officers, 2010), instructional time focused on 
proportionality occurs in middle school when students “develop understanding of proportionality 
to solve single and multi-step problems . . . solve a wide variety of percent problems, including 
those involving discounts, interest, taxes, tips, and percent increase or decrease” (p. 46). 

Solving even simple proportion problems is challenging for many children and 
adolescents when they lack understanding of the problem situation and whether a solution 
strategy is applicable (Adjiage & Pluvinage, 2007; Fujimura, 2001; Lamon, 2007; Lobato et ah, 
2010; Miyakawa & Winslow, 2009; NMAP, 2008; Toumiaire & Pulos, 1985; Weinberg, 2002). 
Although students require instruction that supports the development of underlying concepts and 
flexible procedures to solve proportion problems (NMAP, 2008; Tourniaire & Pulos, 1985), few 
intervention studies have been conducted to improve students' learning of ratios and proportions. 
Most of the existing studies were short-term and did not focus on the broad domain of ratios and 
proportional relationships (Adjiage & Pluvinage, 2007; Fujimura, 2001; Myakawa & Winslow, 
2009) or used quasi-experimental research designs or a teaching experiment, which limited 
causal inferences. Also, few studies have tested the effectiveness of a comprehensive curriculum 
package (e.g., Connected Mathematics Project; see Ben-Chaim, Fitzgerald, Benedetto, & Miller, 
1998) or conducted randomized studies with teachers in various settings in which the 
intervention was implemented with fidelity. 

The few randomized studies conducted to date have examined the efficacy of schema- 
based instruction (SBI), a multicomponent approach to teaching proportional problem solving 
using data from the upper Midwest of the U.S. (e.g., Jitendra et ah, 2009; Jitendra, Star, 
Rodriguez, Lindell, & Someki, 2011; Jitendra, Star, Dupuis, & Rodriguez, 2013; Jitendra et ah, 
in press). The SBI intervention is grounded in schema theory and research on expertise. Four 
major features underlie the SBI approach - priming the mathematical structure of problems, 
using visual representations, explicit teaching of problem solving heuristics, and developing 
procedural flexibility. Recently, Jitendra et al. (in press) described findings of a study in which 
teachers were randomly assigned to either a treatment condition (received professional 
development to implement SBI five days a week for approximately 6 weeks to teach problem 
solving involving ratio, proportion, and percent) or a control condition in which they taught the 
same topics from their district-adopted mathematics textbook. Results indicated that students in 
the SBI classrooms on average outperformed students in the control classrooms on a proportional 
problem solving measure and maintained their improved perfonnance nine weeks later. Scores 
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on the Process and Application subtest of the Group Mathematics Assessment and Diagnostic 
Evaluation (GMADE) were equivalent for the two groups. Thus there was evidence that the SBI 
treatment improves students’ proportional thinking. 


Purpose / Objective / Research Question / Focus of Study: 

The purpose of this study was to replicate and extend the study of the SBI intervention 
conducted by Jitendra et al. (in press) that demonstrated impact in proportional problem solving 
for a homogeneous sample that was predominantly White. While the importance of replication 
in educational research is a staple of methods textbooks, replications are rare in practice 
(Duncan, Engel, Claessens, & Dowsett, 2015) but badly needed (Makel & Plucker, 2014). 

We chose to replicate this study in a geographically diverse location for several reasons: 
(a) there are few validated approaches to instruction in solving a wide range of single and multi- 
step problems involving ratios and proportional relationships, and (b) few replication studies are 
conducted in education (Yong, 2012) even though they are important to scientific inquiry and 
can enhance external validity. Specifically, we examined the following research questions: la. 
What are the effects of the SBI intervention on seventh-grade students’ proportional problem 
solving immediately following the intervention and eight weeks following treatment? lb. What 
are the effects of the SBI intervention on problem solving immediately following the 
intervention and eight weeks following treatment for students receiving special education 
services? 2a. What are the effects of the SBI intervention on seventh-grade students’ general 
mathematical problem solving? 2b. What are the effects of the SBI intervention on general 
mathematical problem solving for students receiving special education services? 3a. Does the 
treatment moderate students’ learning trajectory over time? 

Setting and Population/Participants/Subjects: 

Students from 20 seventh-grade classrooms (N= 429) and their teachers (J = 20) from 10 
middle schools in an urban school district located in the Southeast region of the United States 
participated in the study. Based on available student demographic data for our sample (13% 
missing data), the majority of students were Hispanic (33%), with 27% Black, 21% White, 3% 
Asian, and 3% multiracial. The mean age of these students was 12 years 7 months (SD = 5 
months). Approximately 63% of students were eligible for free or reduced price lunch (FRL), 
21% were English language learners (ELL), and 5% were students receiving special education 
services (see Table 1 for teacher and student demographic infonnation). In sum, 366 of the 429 
students (85%) from all 20 classrooms provided data for three measurement occasions (i.e., 
pretest, posttest, retention test) on the proportional problem solving measure. 

Intervention / Program / Practice: 

The SBI instructional content consisted of ratio, proportion and percent topics. Within 
SBI, teachers used four instructional practices. First, teachers primed the mathematical structure 
of problems by focusing on a variety of problem types related to proportions (see Table 2). 
Teachers were encouraged to stimulate students’ thinking about how problems within and across 
types are similar and different. Second, teachers visually mapped information in the problem 
using schematic diagrams. For example, to visually represent information in a proportion 
problem, teachers prompted students to identify the ratios in the problem and write them in the 
proportion diagram. Third, teachers provided explicit instruction on a problem-solving heuristic 
(DISC: D - Discover the problem type, I - Identify information in the problem to represent in a 
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diagram, S - Solve the problem, C - Check the solution), with accompanying deep-level 
questions for each step in the heuristic (e.g., Why is this a proportion problem?). Finally, 
teachers worked to develop students’ procedural flexibility, including explicit teaching of 
multiple solution methods for solving proportion problems and being cognizant of specific 
methods that are more efficient than others. Teachers modeled by thinking aloud as they engaged 
in these four practices and scaffolded by gradually shifting responsibility for problem solving to 
the students. 

Research Design, Data Collection, and Analysis: 

We used a randomized cluster design with longitudinal (repeated measures) data. For 
each of the 20 teachers, one class of students was randomly selected to participate in the study. 
All students in the participating class that were present at pretest or posttest were included in the 
student sample. Each of the 20 teachers and their participating class was randomly assigned to 
one of two conditions: treatment or control. In treatment classrooms SBI was implemented daily 
for 45-50 min class periods over 6 weeks beginning in late October of the school year. In the 
same time period students in the control condition were taught the same topics as in SBI 
classrooms but using their district-adopted textbook. All treatment teachers participated in 16 
hours of professional development in early October of the school year, approximately three 
weeks before the study period began. The content and focus of the training was on the topics of 
ratio, proportion, and percent, particularly as they related to student understanding and 
implementation of SBI. 

In early October of the school year pretest data were collected. For students this consisted 
of two measures of mathematical problem solving: a researcher-designed test of proportional 
problem solving (PPS) and the Group Mathematics Assessment and Diagnostic Evaluation 
(GMADE) Process and Applications subscale (Pearson Education, 2004), a standardized test of 
mathematical problem solving. Immediately after the intervention students completed the same 
two measures they had completed at pretest. Data were also collected approximately 1 1 weeks 
after the end of the intervention on the PPS test to measure retention of proportional problem 
solving skills. Reliability estimates for the PPS measure at pretest, posttest, and retention were 
0.71, 0.78, and 0.76, respectively. Reliability estimates for the GMADE at pretest and posttest 
were 0.68 and 0.69, respectively. All student measures were group-administered by teachers. 

We gathered fidelity-of-implementation and quality of instruction data via classroom 
observations. We assessed procedural fidelity by observing videotaped lessons using a checklist 
developed to document the presence of the core features (e.g., identifies the problem type by 
focusing on the key problem features, connects the new problem to previously solved problems) 
of the SBI intervention. The same checklist was also used in the control condition to evaluate 
program differentiation and determine whether control teachers provided instruction that was 
similar to the key elements of SBI. The overall quality of instruction in both conditions was 
assessed using four items that focused on features such as the teacher’s ability to clarify the 
lesson purpose, provide lesson closure, manage instructional time (i.e., how well the teacher 
managed student behavior), and minimize mathematical errors. 

Both descriptive and inferential (multilevel) analyses were performed that analyzed the 
posttest and delayed posttest data separately (cross-sectional analyses) as well as the impact of 
treatment on scores across the three repeated measures. Naturally the use of multilevel models 
with 20 clusters raises concerns about whether there was adequate statistical power to detect 
treatment effects. We fitted multilevel models because: (a) previous SBI research has typically 


SREE Spring 2016 Conference Abstract Template 


4 



reported treatment effect sizes of at least .50 SDs for cross-sectional data which produces a 
power of .70 for an intra-class correlation of .15 and a within-cluster sample size of 25; for 
slightly larger effect sizes such as .60 SDs power is .84, (b) it is important to replicate SBI 
findings with a sample that enhances external validity. 

Findings / Results: 

Fidelity-of-implementation results indicate that the mean total score across the seven 
items on the fidelity checklist was 14.45 ( SD = 4.13) for treatment teachers and 9.00 (SD = 4.18) 
for control teachers out of a possible 21 points (higher scores indicate implementation with 
greater fidelity). Results indicated statistically significant differences between the treatment and 
control groups on the total score for fidelity-of-implementation items, with treatment teachers 
implementing SBI elements more than control teachers, in general. With regard to quality of 
instruction, as expected, both treatment (M = 8.36) and control (M = 9.11) teachers were rated 
similarly and, on average, there were no differences between the two groups, t{ 18) = -0.85, p = 
.408. 

Preliminary results of a series of multilevel (i.e., two-level, students within 
clusters/teachers) models for cross-sectional data with covariates at both levels indicated 
statistically significant differences favoring SBI on the PPS posttest (y = 1.69, t(15) = 2.96, p = 
.01, g = 0.44), but no differences on the PPS retention test {p = . 13) or the GMADE posttest (p = 
.08) (see Tables 3, 4, and 5). For the third research question, there was evidence that linear 
slopes over time for PPS varied across students and that this variation was moderated by 
treatment (y = .73, t( 327) = 3.36, p = .001). These findings suggested that the treatment group 
was associated with a larger (positive) linear slope over time compared to the control group. 

Given the small numbers of clusters and its likely impact on power, we calculated effect 
sizes comparing the SBI group with the control group for the nonsignificant findings for the PPS 
retention test and GMADE, which were 0.27 and 0.25 SDs, respectively. Furthennore, results for 
students eligible for special education services indicated effects sizes of 0.56 on the immediate 
PPS posttest, 0.32 on the PPS retention test, and 0.34 on the GMADE. These effect sizes are not 
statistically different from zero but are at or above the .25 “substantively important” threshold 
used by What Works Clearinghouse (2014). Thus it seems likely that our nonsignificant findings 
for the PPS retention test and the GMADE are at least partly due to having only J = 20 clusters. 

Conclusions: 

The consistent effects in previous randomized studies suggest confidence in the positive 
effects of SBI in enhancing proportional problem solving for all students. Using a geographically 
diverse sample, this study replicated and extended earlier findings thus enhancing external 
validity. An SBI effect emerged for the immediate PPS test but unlike the original study (Jitendra 
et ah, in press), participants did not maintain this learning over time for the PPS delayed posttest 
and the GMADE test. Future research, including a larger sample, is needed to confirm for whom 
and under what conditions the SBI intervention has positive effects. We are encouraged, 
however, about the positive effects of SBI compared to regular mathematics instruction for 
students eligible for special education on all measures. 
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Appendix B. Tables and Figures 

Table 1. 

Participant Demographic Information by Treatment 



- 


SBI 



Control 



Total 


n 

% 

M 

SD 

n 

% 

M 

SD 

n 

% 

M 

SD 

Student Information 













Age 




12.73 

0.55 



12.69 

0.55 



12.71 

0.55 

Sex 

Female 

116 

All 



75 

41.0 



191 

44.5 




Male 

100 

40.7 



82 

44.8 



182 

42.4 



Race 

Asian 

4 

1.6 



10 

5.5 



14 

3.3 




Black 

73 

29.7 



44 

24.0 



117 

27.3 




Hispanic 

78 

31.7 



65 

35.5 



143 

33.3 




Multiracial 

8 

3.3 



3 

1.6 



11 

2.6 




White 

53 

21.5 



35 

19.1 



88 

20.5 



FRL 

Yes 

157 

63.8 



114 

62.3 



271 

63.2 




No 

59 

24.0 



43 

23.5 



102 

23.8 



ELL 

Yes 

50 

20.3 



39 

21.3 



89 

20.7 




No 

166 

67.5 



118 

64.5 



284 

66.2 



SWD 

Yes 

17 

6.9 



6 

3.3 



23 

5.4 




No 

199 

80.9 



151 

82.5 



350 

81.6 



Missing 


30 

12.2 



26 

14.2 



56 

13.1 



Teacher Information 













Sex 

Female 

7 

63.6 



7 

77.8 



14 

70.0 




Male 

4 

36.4 



2 

22.2 



6 

30.0 



Math courses taken 



4.45 

3.98 



7.00 

4.84 



5.53 

4.43 

Y ears experience teaching math 



6.45 

5.28 



7.78 

5.83 



7.05 

5.42 


Note. SBI = schema-based instruction; FRL = students eligible for free or reduced priced lunch; ELL = English Language Learner; 
SWD = students with disabilities; Total student N = 429. 
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Table 2 


Examples of Problem Types 


Problem Type 

Ratio 

Compared 


75 

Main line lunches 


100 

Salad bars lunches 


Base 



Example of Problem 


On Thursday, the cafeteria at Osseo Middle School sold: 
42 smoothies, 75 main line lunches, 80 cookies, 51 bags 
of chips, 100 salad bar lunches, and 26 breakfast bars. 
What is the ratio of the number of main line lunches sold 
to the number of salad bar lunches sold on Thursday? 


Percent: Part-whole comparison 




On a chapter test, Janie got a grade of 80%. The test had 
a total of 35 possible points. How many points did Janie 
earn on the test? 


Percent of change 




A tree that was 10 feet tall grew by 5 feet. What percent 
has it grown? 


Proportion 



The Fra nk family from Minnesota, USA, is going to 
Britain for their summer vacation. They exchanged $50 
for 27 British pounds. At that exchange rate, how many 
British pounds could they get for $75? 
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Scale Drawing 


If Then 



On the map of the Disney World Theme Parks, a scale of 
1 inch represents 3 miles. What is the actual distance 
between Magic Kingdom and Animal Kingdom when it 
is 4 inches on the map? 


Percent of change: Sales tax, tips, 
discounts, mark-ups 


Change 


Original 


$200 




Original Change New 


In New York, where the sales tax is 4%, what would be 
the total cost with tax of a leather coat that costs $200? 


Simple interest 


Simple 

Interest 


Principle 





A. 


$700 

& 

/ -$?- \ 

$x 

Principle 

Simple 

Interest 

Balance 


Anna deposits $700 in a savings account at the 
beginning of the year. The simple annual interest rate 
for the savings account is 5%. What will be the balance 
in Anna’s account at the end of the year? 
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Table 3 


HLM Results for PPS Posttest 


Fixed Effect 

B 

SE 

/-value 

df 

P 

Between- Student Model 






Intercept 

16.84 

1.236 

13.63 

15 

<.001 

Gender 

0.40 

0.413 

0.98 

339 

.330 

Asian 

-0.72 

1.144 

-0.63 

339 

.527 

Black 

-0.90 

0.597 

-1.52 

339 

.130 

Hispanic 

-1.05 

0.553 

-1.90 

339 

.058 

Multiracial 

-0.62 

1.236 

-0.50 

339 

.618 

Pretest 

0.58 

0.055 

10.57 

339 

<.001 

Between-Classroom Model 






Treatment 

1.69 

0.571 

2.96 

15 

.010 

SWD 

-0.03 

0.041 

-0.73 

15 

.480 

ELL 

-0.07 

0.034 

-2.17 

15 

.046 

FRL 

-0.05 

0.019 

-2.68 

15 

.018 

Random Effect Variance 

SD 

2 

X 

df 

P 

Classroom 

0.60 

0.773 

234.23 

15 

.003 

Student 

14.31 

3.783 
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Table 4 


HLM Results for PPS Delayed Posttest 


Fixed Effect 

B 

SE 

/-value 

df 

P 

Between- Student Model 

Intercept 

17.09 

1.294 

13.20 

15 

<.001 

Gender 

0.09 

0.401 

0.22 

323 

.827 

Asian 

-0.27 

1.096 

-0.24 

323 

.809 

Black 

-1.55 

0.578 

-2.69 

323 

.008 

Hispanic 

-1.18 

0.530 

-2.24 

323 

.026 

Multiracial 

-2.44 

1.173 

-2.08 

323 

.038 

Pretest 

0.48 

0.054 

8.95 

323 

<.001 

Between-Classroom Model 

Treatment 

0.98 

0.619 

1.58 

15 

.135 

SWD 

-0.06 

0.045 

-1.42 

15 

.178 

ELL 

-0.03 

0.037 

-0.70 

15 

.495 

FRL 

-0.06 

0.020 

-3.28 

15 

.006 

Random Effect 

Variance 

SD 

2 

X 

df 

P 

Classroom 

Student 

0.90 

12.73 

0.951 

3.568 

43.76 

15 

<.001 
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Table 5 


HLM results for GMADE Posttest 


Fixed Effect 

B 

SE 

/-value 

df 

P 

Between- Student Model 

Intercept 

16.55 

1.032 

16.03 

15 

<.001 

Gender 

0.93 

0.389 

2.40 

332 

.017 

Asian 

0.98 

1.060 

0.93 

332 

.355 

Black 

-0.78 

0.562 

-1.39 

332 

.166 

Hispanic 

-0.30 

0.516 

-0.58 

332 

.564 

Multiracial 

-0.32 

1.197 

-0.27 

332 

.788 

Pretest 

0.35 

0.051 

6.89 

332 

<.001 

Between-Classroom Model 

Treatment 

0.88 

0.481 

1.83 

15 

.086 

SWD 

-0.12 

0.036 

-3.46 

15 

.004 

ELL 

-0.04 

0.029 

-1.49 

15 

.158 

FRL 

-0.04 

0.016 

-2.55 

15 

.023 

Random Effect 

Variance 

SD 

2 

X 

df 

P 

Classroom 

Student 

0.29 

12.40 

0.538 

3.523 

28.34 

15 

.019 
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